# Unsupervised Pretraining

SAM2 model based on the HieraDet image encoder, focused on image feature extraction tasks.

A visual feature extraction model developed by NVIDIA for generating image embeddings, supporting downstream tasks such as image classification.

ESM-1b is a Transformer-based protein language model trained via unsupervised learning on protein sequence data, capable of predicting protein structure and function.

Wav2Vec2 is a self-supervised learning-based speech recognition model, pre-trained and fine-tuned on 960 hours of LibriSpeech audio data, supporting English speech transcription.

This model is used to evaluate the relevance and specificity of the last message in a dialogue, based on the sberbank-ai/ruRoberta-large architecture.

A Wav2Vec2 base model pretrained on the Dutch subset of the VoxPopuli corpus, suitable for Dutch speech recognition tasks.

A base Wav2Vec2 model pretrained on 100 hours of unlabeled Vietnamese speech audio from the VLSP dataset, requiring fine-tuning for downstream tasks.

T5 1.1 is Google's improved text-to-text transfer model, utilizing the GEGLU activation function and optimized architecture, focused on unsupervised pretraining

A large-scale speech recognition model pretrained on the VoxPopuli French corpus, supporting French speech-to-text tasks

T5 1.1 is Google's improved text-to-text transfer Transformer model, utilizing GEGLU activation function and optimized architecture, pretrained solely on the C4 dataset in an unsupervised manner